Papers with causal intervention techniques
Causal interventions expose implicit situation models for commonsense language understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Classical psycholinguistic accounts have suggested that world knowledge enters into language understanding through structured schemas called situation models. |
| Approach: | They apply causal intervention techniques to transformer models to analyze performance on the Winograd Schema Challenge . |
| Outcome: | The proposed model performs well on the Winograd Schema Challenge . |
A Causal View of Entity Bias in (Large) Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Entity bias affects pretrained (large) language models, causing them to rely on (biased) parametric knowledge to make unfaithful predictions. |
| Approach: | They propose a structured causal model whose parameters are easier to estimate . they propose to perturb the original entity with neighboring entities . |
| Outcome: | The proposed model reduces biasing information pertaining to the original entity while still preserving sufficient semantic information from similar entities. |